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ABSTRACT 



Analysis of variance (ANOVA) designs typically use what is 
referred to as crossed design to test for differences between means of 
groups. In a balanced, or crossed, one-way ANOVA, each student (unit of 
analysis) would have a score in each of the experimental conditions. In a 
two-way design, the analysis is considered crossed if each level from one way 
is contained in each level of the other way. In this design, every person 
(unit of analysis) has a score in every cell. Many experimental designs in 
the behavioral sciences do not qualify as a crossed design. Units of 
measurement are rather "nested" inside other factors. A crossed experimental 
design would neglect the hierarchical structure of the data and produce 
incorrect interpretations of results. Neglecting a nested design when one 
actually exists will make the researcher: (1) wrongly attribute a main effect 

to an interaction effect when, in fact, no interaction exists; (2) divide by 
the wrong degrees of freedom when determining the mean square and F-value 
(and the statistical significance of the F-value); and (3) assume that a main 
effect has a smaller effect size (eta-squared) because the sum of squares for 
that effect is being partly attributed to the interaction effect. (SLD) 
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Nested ANOVA vs. crossed ANOVA: When and how to use which 
“Crossed” One-Way ANOVA 

ANOVA designs typically use what is referred to as a “crossed design” to test for 
differences between means of groups. In a crossed (or balanced) one-way ANOVA 
design, a researcher would expect each subject to have a score in each cell. For example, 
a teacher may want to test the differences in results obtained from three reading 
strategies. The teacher would record scores for each of his or her students in each of the 
experimental (reading strategies) conditions. The analysis would be a balanced, or 
crossed, one-way ANOVA design because each student would have a score in each of the 
experimental conditions. 

“Crossed” Two-Way ANOVA 

In a two-way design, the analysis is considered crossed if each level from one 
way is contained in each level of the other way. In this design, every person (assuming 
persons are the unit of analysis) has a score in every cell. For example, suppose a 
researcher was interested in the effects of three different dosages of vitamins 
administered in two different forms (oral and injection) to children. In this example, the 
first way would have three levels (dosages) and the second way would contain two levels 
(oral or injection). This ’would yield a 3 X 2 ANOVA with six (3*2 = 6) cells in the 
design. For this experiment, each child would have 6 scores, one in each cell. This type 
of design, illustrated by the crosstabulation in Table 1, is referred to as a crossed design 
because each person has a score in every cell. 



Insert Table 1 about here 
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Nested ANOVA Designs 

Many experimental designs in the behavioral sciences do not qualify as a crossed 
design. Units of measurement are, rather, “nested” inside other factors. Consider the 
following examples of nested designs: students nested within classrooms; students 
nested within schools; and students nested within classrooms nested within schools. In 
each of the occasions listed above, a crossed experimental design would neglect the 
hierarchical structure of the data and would produce incorrect interpretations of results. 

To illustrate the effects of neglecting nested structure, a hypothetical data set of 
student reading scores has been created. In this example, student scores are obtained in 
five teachers’ classrooms within each of four different schools. If the data from this 
experiment were not correctly treated as a nested design, the analysis would entail a 
crossed 4X5 ANOVA. Table 2 shows what the data structure would look like if this 
crossed analysis were performed. Table 3 illustrates the results of the crossed data 
ANOVA. 



Insert Tables 2 and 3 about here 



Although not previously stated, a nested structure is implicit in this design. It 
stands to reason that each of the five teachers does not teach in each of the four schools, 
but that they only tech in one school only. Recognizing this hierarchical structure, we 
treat each teacher as a unique factor within each school. Therefore, instead of a 4 X 5 
ANOVA, we have a 4 X 20 ANOVA with four schools in the first way and twenty 
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teachers in the second way. Table 4 illustrates the revised structure of this data. It should 
be noted, before the ANOVA is run, that there is no interaction effect in this data set 
since each teacher is in one and only one school and each student is in one and only one 
classroom. 



Insert Tables 4 and 5 about here 



Table 5 shows the results from the 4 X 20 ANOVA of the data in Table 4. Notice 
that the sum of squares (SS) for the teacher main effect in Table 5 is exactly equal to the 
combined SS of the teacher main effect and school*teacher interaction effect from Table 
3. This illustrates how a researcher could fail to reject a null hypothesis simply because 
the data were incorrectly structured. 

What Happens When I Use a Crossed Design and Should Have Used a Nested Design? 

Although recognizing the structure of the data will help to yield accurate results, 
one price that is paid for performing a nested design is that of confounded results 
(Lindman, 1992). In this example, the teacher main effect and school*teacher interaction 
effect are considered confounded. The issue of confounded effects plays a larger role in 
interpretation of results than in the actual statistical test (Lindman, 1 992). In this 
example, the large eta squared for the schools main effect is likely to be interpreted as 
differences between factors that are independent of teachers (e.g., administration and 
ethnic make-up of students). This would lead the researcher to assume that the teacher 
main effect is testing the difference between the mean scores of students between 
classrooms, independent of schools. In a nested design, however, the teacher main effect 
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is somewhat dependant on the school main effect because not all teachers were randomly 
assigned to all schools. Thus, the results are confounded and the large eta squared for the 
teacher main effect could be due, in part, to the differences between children within 
schools. 

For example, it could be that teacher 5 in school 1 (mean of 2.5) is actually a 
better teacher than teacher 6 in school 2 (mean of 4) and the difference in mean score is 
simply due to the type of school that the teachers are in. The teacher main effect would 
not detect this difference, but would test only for the differences between the means of 
the 2 students within each classroom unique to that level of the school way. In other 
words, the results from the ANOVA in Table 5 tells us that we should reject the null 
hypothesis that the means of the students’ scores in teachers’ classrooms within each 
level of the school way are equal. For a more detailed description of interpreting results 
from nested ANOVA designs see Hicks (1973) and Lindman (1992). 

Summary of Findings: Neglecting a nested design when one exists will make the 
researcher . . . 

1 . ... wrongly attribute a main effect to an interaction effect when in fact no 
interaction exists. 

2. ... divide by the wrong degrees of freedom when determining the mean square 
and F-value (also the statistical significance of the F-value). 

3. ... assume that a main effect has a smaller effect size (eta-squared) because the 
sum of squares for that effect (in this case the teacher main effect) is being partly 
attributed to the interaction effect (in this case the school*teacher interaction 
effect). 
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Table 1 

Hypothetical crosstabulation of a completely crossed 3X2 design with 4 subjects 



Oral 



Injection 



Dosage 1 Dosage 2 Dosage 3 



Subject 1 score 
Subject 2 score 
Subject 3 score 
Subject 4 score 


Subject 1 score 
Subject 2 score 
Subject 3 score 
Subject 4 score 


Subject 1 score 
Subject 2 score 
Subject 3 score 
Subject 4 score 


Subject 1 score 
Subject 2 score 
Subject 3 score 
Subject 4 score 


Subject 1 score 
Subject 2 score 
Subject 3 score 
Subject 4 score 


Subject 1 score 
Subject 2 score 
Subject 3 score 
Subject 4 score 
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Table 2 

Data for a crossed design of the reading data 



Teacher 


1 


2 


School 

3 


4 




8 


4 


10 


8 


1 




7 


4 


10 


8 




7 


4 


10 


8 


2 




6 


5 


8 


8 




6 


5 


9 


10 



5 


5 


8 


10 


5 


6 


6 


10 





5 


7 


7 


10 




3 


8 


7 


11 


5 


2 


8 


6 


12 
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Table 3 



ANOVA for crossed data 



Source 


SS 


df 


MS 


F 


Sig. 


Eta 

Squared 


School 


118.900 


3 


39.633 


113.238 


.000 


.558 


Teacher 


.850 


4 


.212 


.607 


.662 


.004 


School*Teacher 


86.350 


12 


7.196 


20.560 


.000 


.405 


Error 


7.000 


20 


.350 








Total 


213.100 


39 
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Table 4 

Data for a nested design of the reading data 







School 






Teacher 


1 


2 


3 


4 


1 


8 








7 








2 


7 

6 








3 


6 

5 








4 


5 

5 








5 


3 

2 








6 




4 

4 






7 




4 

5 






8 




5 

5 






9 




6 

7 






10 




8 

8 






11 






10 

10 




12 






10 

8 




13 






9 

8 




14 






6 

7 




15 






7 

6 




16 








8 

8 


17 




• 




8 

8 


18 








10 

10 


19 








10 

10 


20 








11 

12 
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Table 5 



ANOVA for nested data 



Source 


SS 


df 


MS 


F 


Sig. 


Eta 

Squared 


School 


118.900 


3 


39.633 


113.238 


.000 


.558 


Teacher 


87.200 


16 


5.450 


15.571 


.000 


.409 


SchooPTeacher 












Error 


7.000 


20 


.350 








Total 


213.100 


39 
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